1. Field of the Invention
The present invention relates to information processing units, information processing methods, and recording media therewith. More particularly, the present invention relates to an information processing unit, an information processing method, and a recording medium therewith, for accurately predicting subsequent motion by storing motion information and torque information in advance and predicting the subsequent motion based on the stored motion information and torque information.
2. Description of the Related Art
Hitherto, conventional systems for measuring or estimating motion parameters of an object based on an input of an image or a sensor output fail to accurately estimate these parameters due to noise included in the input data and the inadequacy of data relative to the parameters to be estimated.
To this end, Kalman filters described in xe2x80x9cA New Approach to Linear Filtering and Prediction Problemsxe2x80x9d by R. E. Kalman, Trans. ASME-J. Basic Eng., pp. 35-45 March 1960, or recursive estimation filters derived from the Kalman filters have been used.
In linear systems, Kalman filters are often used as the recursive estimation filters. In nonlinear systems, extended Kalman filters (hereinafter referred to as xe2x80x9cEKFxe2x80x9d) are the most popular. The EKF is described in xe2x80x9cApplied Optimal Estimationxe2x80x9d by A. Gelb, The Analytic Science Corp., 1974. Other applications of Kalman filters include unscented filters described in xe2x80x9cA New Extension of the Kalman Filter to Nonlinear Systemsxe2x80x9d by S. J. Julier, Proc. of AeroSense: The 11th International Symposium on Aerospace/Defense Sensing, Simulation and Control, and pseudo-Kalman filters described in xe2x80x9cUsing Pseudo Kalman Filters in the Presence of Constraints, Application to Sensing Behavioursxe2x80x9d by T. Vileville and P. Sander, Technical Report 1669, INRIA-Sophia, Valbonne, France, 1992.
FIG. 11 is a block diagram of a Kalman filter 120. The Kalman filter 120 employs an efficient least-square method. The Kalman filter 120 estimates states in the past, present, and future. The Kalman filter 120 is capable of estimating state variables based on incomplete information. In addition to the state variables to be estimated, the Kalman filter 120 updates simultaneously error covariance indicating estimation accuracy. Use of the Kalman filter 120 is known as a robust estimation method.
The Kalman filter 120 is one of the recursive estimation filters. The Kalman filter 120 always exclusively uses the most recent data from observed values input in series. The Kalman filter 120 includes a predicting unit 131 for predicting motion and an updating unit 132 for updating the predicted data. Estimation is carried out by alternatively repeating prediction and updating of the data by, respectively, the predicting unit 131 and the updating unit 132.
Referring to FIG. 12, operation of the Kalman filter 120 is described. A set of state variables to be estimated is defined as a state vector x. When the number of the state variables is n, the state vector x is n-dimensional. A k-th cycle (one cycle corresponds to one observation cycle) state vector x is expressed by a state vector Xk. A prediction process for predicting a subsequent state is expressed by an expression (1). In expressions illustrated hereinafter, a vector is expressed with an arrow at the top thereof; a first order time differential of a vector is indicated by a dot ({dot over ( )}) at the top thereof; and a second order time differential is indicated by two dots at the top ({dot over ( )}) thereof:
{right arrow over (xk+1)}=Ak{right arrow over (xk)}+{right arrow over (wk)}xe2x80x83xe2x80x83(1)
In the above expression, a matrix Ak is the k-th cycle state transition matrix with n rows and n columns. wk represents prediction process noise generated in the prediction process. It is assumed that wk is normally distributed Gaussian noise.
An observation process is expressed by an expression (2):
{right arrow over (yk)}=Hk{right arrow over (xk)}+{right arrow over (vk)}xe2x80x83xe2x80x83(2)
The covariance matrix of the process noise generated in the process shown by the expression (1) is expressed by Qk.
In the expression (2), yk indicates an observation vector representing a set of the k-th cycle observed values, and Hk represents the k-th cycle observation matrix. The observation vector yk does not necessary have the same number of dimensions as the state vector. If the observation vector yk is assumed to be m-dimensional, the observation matrix Hk has m rows and n columns. vk represents a vector of observation noise generated in the observation process, which is assumed to be Gaussian noise. The covariance matrix of the observation noise is expressed by Rk.
The Kalman filter 120 performs estimation through repetitions of predicting and updating. The estimated state vector before the updating is expressed by an expression (3), and the estimated state vector after the updating is expressed by an expression (4):
{right arrow over (X{overscore (k)})}xe2x80x83xe2x80x83(3)
{right arrow over (Xk)}xe2x80x83xe2x80x83(4)
Let Pk represent the estimated error covariance after the updating. The estimated error covariance before the updating is expressed by an expression (5):
Pxe2x88x92kxe2x80x83xe2x80x83(5)
Assuming the above, processing of the predicting unit 131 is described. As shown in FIG. 12, the predicting unit 131 performs process e1 to predict the next state based on an expression (6):
{right arrow over (X{overscore (k)})}+1=Ak{right arrow over (Xk)}xe2x80x83xe2x80x83(6)
The predicting unit 131 performs process e2 to predict the next estimated error covariance Pxe2x88x92k+1 based on an expression (7):
xe2x80x83Pxe2x88x92k+1=AkPkAkT+Qkxe2x80x83xe2x80x83(7)
In the above expression, AkT represents a transposed matrix of the state transition matrix Ak.
Next, processing of the updating unit 132 is described. The updating unit 132 performs process c1 to compute a Kalman gain Kk based on an expression (8):
Kk=Pxe2x88x92kHkT(HkPxe2x88x92kHkT+Rk)xe2x88x921xe2x80x83xe2x80x83(8)
In the above expression, HkT represents a transposed matrix of the observation matrix Hk, and ( )xe2x88x921 represents an inverse matrix of the matrix within the parentheses ( ).
The updating unit 132 performs process c2 to update the estimated value of the state variable based on the input observed value using an expression (9):
{right arrow over (Xk)}={right arrow over (X{overscore (k)})}+Kk({right arrow over (yk)}xe2x88x92Hk{right arrow over (X{overscore (k)})})xe2x80x83xe2x80x83(9)
The updating unit 132 performs process c3 to update the estimated error covariance Pk based on an expression (10):
Pk=(Ixe2x88x92KkHk)Pxe2x88x92kxe2x80x83xe2x80x83(10)
In the above expression, I represents a unit matrix that has nxc3x97n dimensions.
The above-described Kalman filter 120 assumes that both the prediction process and the observation process are linear. In practice, however, the prediction process and the observation process are often nonlinear. To accommodate such nonlinear processes, various types of derived filters including the EKF have been proposed. The EKF, which is most commonly used, is described below.
The EKF expresses a prediction process as an expression (11) (corresponding to the expression (1) of the Kalman filter 120), and an observation process as an expression (12) (corresponding to the expression (2) of the Kalman filter 120):
{right arrow over (xk+1)}=f{right arrow over ((xk))}+{right arrow over (wk)}xe2x80x83xe2x80x83(11)
{right arrow over (yk)}=h{right arrow over ((xk))}+{right arrow over (vk)}xe2x80x83xe2x80x83(12)
In the expression (11), f represents a nonlinear state transition matrix. In the expression (12), h represents a nonlinear observation matrix.
In the EKF, the actual estimation framework is expressed by an expression (13) which corresponds to process e1 of the Kalman filter 120 (FIG. 12):
xe2x80x83{right arrow over (X{overscore (k)})}+1=f{right arrow over ((Xk))}xe2x80x83xe2x80x83(13)
In the EKF, processing corresponding to process c2 of the Kalman filter 120 (FIG. 12) is expressed by an expression (14):
{right arrow over (Xk)}={right arrow over (X{overscore (k)})}+Kk({right arrow over (yk)}xe2x88x92h{right arrow over ((X{overscore (k)})})xe2x80x83xe2x80x83(14)
In the EKF, a state transition matrix Ak is expressed by an expression (15), and an observation matrix Hk is expressed by an expression (16):                               A                      k            ⁢                          xe2x80x83                        ⁢                          (                              i                ,                j                            )                                      =                                            ∂                              f                                  (                  i                  )                                                      ⁢                          xe2x80x83                        ⁢                                          (                                  X                  k                                )                            →                                            ∂                          x                              k                ⁡                                  (                  j                  )                                                                                        (        15        )                                          H                      k            ⁢                          xe2x80x83                        ⁢                          (                              i                ,                j                            )                                      =                                            ∂                              h                                  (                  i                  )                                                      ⁢                          xe2x80x83                        ⁢                                          (                                  X                  k                                )                            →                                            ∂                                          x                                  k                  ⁡                                      (                    j                    )                                                              →                                                          (        16        )            
In the expression (15), Ak(i,j) represents a matrix element in the i-th row, j-th column, of the state transition matrix Ak, which is computed by partially differentiating the i-component of a function f by the j-component of the state vector xk. In the expression (16), Hk(i,j) represents a matrix element in the i-th row, j-th column, of the observation matrix Hk, which is computed by partially differentiating the i-component of a function h by the j-component of the state vector xk.
Except the points described above, the EKF operates in the same manner as the Kalman filter 120.
In the above-described conventional recursive filters, the prediction process (represented by the expression (1) or the expression (2)) employed for prediction assumes constant angular velocity, constant angular acceleration, constant moving velocity, or constant moving angular acceleration.
For example, the angular acceleration is assumed to be constant in publications such as xe2x80x9cMeasurement of Angular Velocity and Angular Acceleration in Sports Using Extended Kalman Filtersxe2x80x9d by K. Ohta and K. Kobayashi, Collected Papers of the Society of Instrument and Control Engineers, vol. 31, no. 9. By way of another example, the moving acceleration is assumed to be constant in publications such as xe2x80x9cTracking Hand Dynamics in Unconstrained Environmentsxe2x80x9d by Y. Azoz, IEEE Automatic Face and Gesture Recognition, pp. 274-279, April 1998. These assumptions do not sufficiently model the physical properties of an object. Therefore, the following problems may arise:
1. When inputting an image, ambiguous information on the depth direction may not be recovered.
2. Estimation errors may occur due to inadequacy of information for predicting the motion of a fast-moving object between two successive images or between successively sampled sensor information.
These problems are described below by way of examples. Referring to FIG. 13, it is assumed that an arm 142 of an object (a person) 141 is observed from the p-direction (direction from the right side of the object 141) and from the i-direction perpendicular to the p-direction. Joints include a wrist joint 142-1, an elbow joint 142-2, and a shoulder joint 142-3. Referring to FIGS. 14A and 14B and FIGS. 15A to 15E, examples of the first problem described above are illustrated. FIGS. 14A and 14B are illustrations of the arm 142 of the object 141 observed from the i-direction. Continuous movement of the arm 142 is illustrated by FIGS. 14A and 14B, moving from image i1 shown in FIG. 14A to image i2 shown in FIG. 14B. Image p1 in FIG. 15A and image p2-a in FIG. 15B illustrate images of the arm 142 of the object 141 captured from the p-direction, when images i1 and i2 in FIGS. 14A and 14B are observed. Even if the data of image p1 in FIG. 15A is accurately estimated for image i1 in FIG. 14A, when the prediction process is inappropriate, the actual position indicated by the data of image i2 in FIG. 14B will not be estimated correctly for image p2-a in FIG. 15B. Instead, the actual position may be erroneously estimated as image p2-b in FIG. 15C, image p2-c in FIG. 15D, or image p2-d in FIG. 15E.
Referring now to FIGS. 16A and 16B, an example of the second problem described above is illustrated. When an image moves from the state shown in FIG. 16A to the state shown in FIG. 16B, the magnitude of movement is great. The conventional method fails to accurately predict the second state (FIG. 16B) based on the first state (FIG. 16A) due to inadequacy of information.
Accordingly, it is an object of the present invention to provide an information processing unit, an information processing method, and a recording medium therewith, to accurately predict motion of an object.
In accordance with a first aspect of the present invention, there is provided an information processing unit including an information storage unit for storing motion information and torque information of an object and a motion estimating unit for estimating motion based on the information stored by the information storage unit.
In accordance with another aspect of the present invention, there is provided an information processing method including an information storing step for storing motion information and torque information of an object and a motion estimating step for estimating motion based on the information stored in the information storage step.
In accordance with another aspect of the present invention, there is provided a recording medium for recording a computer readable program. The program includes an information obtaining step for obtaining motion information and torque information of an object and a motion estimating step for estimating motion based on the information obtained in the information obtaining step.
Preferably, in the information processing unit and the information processing method, the motion information and the torque information of the object are stored, and the motion is estimated based on the stored information.
Preferably, in the program recorded on the recording medium, the motion information and the torque information of the object are obtained, and the motion is estimated based on the obtained information.